Some Theorems Concerning the Free Energy of (Un)Constrained Stochastic Hop eld Neural Networks

General stochastic binary Hop eld models are viewed from the angle of statistical mechanics. Both the general unconstrained binary stochastic Hop eld model and a certain constrained one are analyzed yielding explicit expressions of the free energy. Moreover, conditions are given for which some of these free energy expressions are Lyapunov functions of the corresponding di erential equations. In mean eld approximation, either stochastic model appears to coincide with a speci c continuous model. Physically, the models are related to spin and to Potts glass models. By means of an alternative derivation, an expression of a `complementary' free energy is presented. Some surveying computational results are reported and an alternative use of the discussed models in resolving constrained optimization problems is discussed. 1 Unconstrained Stochastic Hop eld Networks 1.1 The Background: Classical Hop eld Networks In 1982 Hop eld introduced the idea of an `energy function' into neural network theory using an asynchronous updating rule and binary units [6]. He used the following expression of the energy: E(S) = 1 2 X ij wijSiSj IiSi; (1) where S 2 f0; 1g is the state vector (S1; ; Sn) of the neural network, Si the output value and Ii the external input of neuron i, and where wij represents the interconnection strength from neuron j to neuron i. In 1984, he generalized the stochastic model to a deterministic one with continuous-valued units [7]. Hop eld used the well known updating rule _ Ui = @EHM (V) @Vi = X j wijVj + Ii Ui; (2) keeping always Vi = g(Ui). Essentially, this is a parallel gradient descent method. 1 There exist other algorithms to nd an equilibrium point of the neural network like V new i = g( P j wijV old j + Ii). However, those are not analyzed here. The corresponding energy function is EHM (V) = 1 2 X